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Abstract 

A multiterminal lossy coding problem, which includes various problems such as the Wyner-Ziv problem and 
the complementary delivery problem as special cases, is considered. It is shown that any point in the achievable 
rate-distortion region can be attained even if the source statistics are not known. 
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I. Introduction 

Recently, the authors investigated the following coding problem Consider a coding system composed 
of one encoder and J decoders. The encoder observes the sequence generated by a memoryless source with 
generic variable X. Then, the encoder broadcasts the codeword to the decoders over the noiseless channel with 
capacity R. The purpose of the j-th decoder is to estimate the value of the target source Zj as accurately as 
possible by using the side information Yj and the codeword sent by the encoder, where {Yj}j =l and {Zj}j =1 
may be correlated with X. Accuracy of the estimation of the j-th decoder is evaluated by some distortion 
measure Sf* and it is required that the expected distortion is not greater than the given value Aj. Fig. [T]depicts 
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qq | the coding system where J = 3. 



O 



X 



In |Q], we proposed a coding scheme which is universal in the sense that it attains the optimal rate-distortion 



■^J- . tradeoff even if the probability distribution Px of the source X is unknown, while the side informations {Yy} J=1 

and the targets {Zj}j =1 are assumed to be generated from X via a known memoryless channel. In Ql, we 
considered only stationary and memoryless sources. In this paper, we extend the result of H) to the case where 
sources are stationary and ergodic sources. 

As mentioned in |fl~), our coding problem described above includes various problems as special cases. For 
example, the Wyner-Ziv problem, i.e. the rate-distortion problem with side information at the decoder Q, is a 
special case of our problem, where J = 1 and Z\ = X. A variation of the Wyner-Ziv problem, where the side 
information may fail to reach the decoder |3]-||3, is also included as a special case, where J = 2, Y\ = 0, 
Y2 = Y and Z\ = Z^ = X (see Fig. [2j. Moreover, our coding system can be considered as a generalization 
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of the complementary delivery (6), Q. In fact, a simple complementary delivery problem depicted in Fig. [3] 
is the case where J — 2, X — {Xx,X2}, Y\ — X2, Yi — X\, and Zj = Xj (j — 1. 2). Further, our coding 
problem includes also the problem considered in [8| (depicted in Fig. [4]i as a special case, where J = 2, 
X = {X ,X 1 ,X 2 }, Yj = X, (j = 1, 2), and Z t = Z 2 = X . 

II. Main Result 

At first, we introduce some notations. We denote by N the set of positive integers {1, 2, . . . }. For a set A and 
an integer n G N, A n denotes the rt-th Cartesian product of A. For a finite set A, \A\ denotes the cardinality 
of A. Throughout this paper, we will take all log and exp to the base 2. 

Let X = X1X2 ... be a stationary and ergodic source with finite alphabet X. For each n G N, X n denotes 
the first n variables (Xi, X2, ■ ■ ■ , X„) of X and the distribution of X n is denoted by Px™- 



Fix J G N. We consider random variables Yj (resp. Zj) taking values in sets 3^ (resp. Zj) where j ranges 

'j and Zj 



over the index set J = {1,2, . . . , J}. We assume that, for each j G J ', yj and Zj are finite sets. We write 



yj = ]\yj, yj = {y 3 }jej 

and 

Let W: X — >y>jxZjbea transition probability. In the followings, we assume that W is fixed and 
available as prior knowledge. For each n G N, let W n be the n-th extension of W, that is, 



W n {y n J ,z n J \x n )^J{w{yj >i ,z Sti \x i ) 

for any sequences 



A 



x n = (xi,x 2 ,. ■ ■ ,x n ) G X n , 

yj = (yj,i,yj,2,---,yj, n ) cy}, and 

Z J = ( Z JA, Z J,2, ■ ■ ■ , Zj,n) S Zj. 

Then, by a source X and a transition probability W, sources Yj = {Yj^}°^ 1 and Zj = {Zjj}'^ 1 are 
induceq^. In other words, Yj = {Yjj, . . . , Yj^ n ) (resp. Zj = (Zj,i, ■ ■ ■ , Zj tn )) is a random variable on 
(resp. Zj) such that 

Px~Y S z-{x n ,yj,zj) 
= P x ~(x n )W n (yj,Zj\x n ) 

n 

= Px4x n )I[W(yj, i ,zj, i \x i ) 

'Note that the transition probability W is stationary and memoryless, while the source X is stationary and ergodic. Further, we assume 
that W is known both to the encoder and decoders, while X is unknown. Universal Wyner-Ziv coding in a setting similar to ours is 
considered in |9|. 
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for any x n 6 X n , y n j G y Jt and z n j G Z^. For each j G J, lj = ( res P- ^ = {^.Jti) is called 

the j'-th component of Yj (resp. Zj)- Note that the joint distribution Px^Ypzj of X n , YJ 1 = (Y^,i, . . . , Yj in ), 
and Z™ = (Zj,i, . . . , Zj )Tl ) is given as a marginal distribution of Px"Yjz™, that is, 



3¥i wj|.*3 B 



for any x n G A"", y™ G and z™ G Z™, where the summation is over all (yj, Zj) G ^ x Zj such that 
the j-th component is (y", z"). 

Further, for each j G J7", let Z,- be a finite set. Then, the formal definition of a code for our coding system 
is given as follows. 

Definition 1: An n-length block code 



is defined by mappings 



and 



C n = (cp n ,^\...,^) 
4> n : X n ->{l,2,...,M n } 
: {1, 2, . . . , M„} x y? Z", Vj G J. 



</>„ is called the encoder and ^4"^ is called the j-f/z decoder. 

The performance of a code C„ = (</>„, ipn \ ■ ■ ■ i^n " 1 ) is evaluated by the coding rate and the distortion 
attained by C n . The coding rate of C n is defined by (l/n)log ||<^ n ||> where \\4> n \\ is the number M n of the 
codewords of C n . For each j G J', let 

(i) 

be a distortion measure, where d^Lx < oo. Then, for each n G N, the distortion between the output z™ = 
(zj,i, ■ ■ ■ , Zj iTl ) G Z™ of the j-th decoder and the sequence z™ = (zj t i, . . . , Zj, n ) 6 Z™ to be estimated is 
evaluated by 

n 

dUHz?,z?)±-'£d? ) (z j , i ,z j , i ). 

1=1 

Definition 2: A pair (R, Aj) of a rate R and a J-tuple Aj = (Ai, . . . , A,/) of distortions is said to be 
achievable for a source X if the following condition holds: For any e > and sufficiently large n there exists 
a code C n = (<j> n , ip£\. . . , ipn ] ) satisfying 

-log \\4> n || < R + e 



and, for any j G J7\ 



< A, + e 



where Exnypz" denotes the expectation with respect to the distribution Px^Ypz^- 

Now, we state our main result. The theorem clarifies that, whenever (R, Aj) is achievable, (R, Aj) is also 
achievable universally. 
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Theorem 1: For given (R,Aj) and S > 0, there exists a sequence {Cn}^ =1 of codes which is universally 
optimal in the following sense: For any source X for which (R, A j) is achievable there exists uq — no(S, X) 
such that, for any n > no, C n — (4> n ,">Pn ■, ■ ■ ■ > V'n ) satisfies 



— log 

n 



<R + S 



and 



Ex., 



< Aj +6 



for any j E J. 

The proof of the theorem will be given in the next section. 

III. Proof of TheoremQ] 
Let (R, A j) and S > be given. Fix e > satisfying 

4 Je + 2eD maK < S 

where 

-Dmax = ma x4 J iL- 

For each n 6 N, let fc„ = log log n. 

Let C„ be the set of all n-length block codes C„ = (<p n , rp^ , . . . , ip { n J) ) such that \\<p n \\ < 2 n( - R+t \ Then, 
let T> n be the set of J-tuple (tprP , ■ • ■ , ">pn J ^ ) of decoders such that (</> n ,^4 , •••>'0ra ) S C„ for some </> n . 
Note that for I e N, 



Mi < n ^ 



(2« H +«>|;y 3 -|') 



(1) 



For each j e J, a sequence x n £ Af n , and a code C„ = (0„, rpn , •••■> )> let 



A 



It should be noted that, by using d^ (x n , C n ), the average distortion attained by the code C n can be written as 



E 



(2) 



For I (1 < I < n) and s (0 < s < /), let <j; :s be the non-overlapping empirical distribution of a;" defined as 



<?; ;s (a J |a;") = 



{0<i< L(n- S )AJ:4i+ s +S = « fc } 



L(n - s)/ZJ 
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For each n £ N, let Q n be the set of all sequences x n £ X n satisfying the following condition: There are 
an integer I (1 <l < k n ) and a code C; £ Ci such that for some integer s (0 < s < I), 

J2 M a V)^V,Q) < A J + 4Jc, Vj£j. (3) 

Now, we describe the construction of the code (7 n = (cj) ni ^\ . . . , "tpn)- 
m Encoder <j> n : The encoder encodes a given sequence x™ £ X n as follows. 

1) If x n £ £/„, then choose integers I, s and a code C; satisfying (0. If x n £ Q n then error is declarecj^. 

2) Send Z and s by using 2 log k n bits. 

3) Send the index of decoders (ipj , . . . , ipi^) £ 29; by using log |Z?j| bits. 

4) Send the codewords 4 > i{ x u+i+t S ) °f blocks x^ti?^ s (0 < i < \_(n — s)jl\) encoded by fa. 
* Decoder 'ipn : The j'-the decoder decodes the received codeword as follows. 

1) Decode the first 21ogfc„ bits of the received codeword and obtain I and s. 

2) Decode the first log \T>i \ bits of the remaining part of the received codeword and obtain the decoders 
(ipl^ , • ■ • , V';' 7 '') G 29; chosen by the encoder. 

(i) 

3) Decode the remaining part of the received codeword by using ijjf and the side information y™. Then, 
the blocks z^Y^t^" (0 < i < [_(n — s)/l\) are obtained. 

The remaining part of the output z n , i.e. zf and zj( n _ g \ is defined arbitrarily. Note that the total 
length of zf and z?, n _ s y l , l+1+s is at most 21. 

A. Optimality of the Code 

By the fact that 1 < I < k n and T>i satisfies (Q~|i, it is easy to see that the coding rate (1/n) log \ \<fin\\ of C n 
satisfies 

-log||0„|| < R + S 
n 

for sufficiently large n. Hence, to show the optimality of the code C n , it is sufficient to bound the distortion 
attained by C n . 

At first, suppose x n £ Q n . By (0) and the additivity of the distortion measures, the code C n satisfies that 

l(n-s)/H „, 



d^fa" C 1 < - V (x (l+1)l+s C, 

a n [x , l,„j^ l( n -s)/l\ 1 \ il + 1 + s ' 1 

21d U) 



< A,- + 4Je 



9 A- d (i) 



11 

for any j £ J. 

On the other hand, if x 11 ^ (?„ then the error is declared (and the codeword is chosen arbitrarily). In this 
case, the distortion occurred at the j-th decoder is upper bounded by dmax- 

2 In this case, the encoder may choose a codeword arbitrarily and send it to the decoders. The choice of the codeword, which is sent 
when the error is declared, does not affect the analysis of the distortion. 
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Hence, for any j G J, we have 

= Px*{x n )d${x n ,C n ) 



Px<x n W{x n ,C n ) 



< J2 ^(O A,+4Je 



9 k 



+ J2 Px»W)d£L 

< Aj + 4Je + + P x „ ((g) d«L 

< A . + 4 j e + 2fc "^ max + P Xn ($<•) D max (4) 

where Q\ denotes the complement of Q n . 

Further, as shown in Lemma [TJ in the appendix, if (R,Aj) is achievable for X then 

(Qn) < e (5) 



P 



X r ' 



holds for sufficiently large n. 
Hence, for sufficiently large n, 

Px~ {x n )d$ (x n ,C n ) < Aj + 4Je + 2eD max (6) 

X n 

By (O and (|6), the average distortion attained by C n = ($ n i^Pn \ ■ ■ ■ i ^n ^) is bounded as 



E, 



<Aj+S 



for any j G J\ This completes the proof of Theorem [TJ 



Appendix 

Lemma 1: Let X be a stationary and ergodic source for which (R, A j) is achievable. Then, for sufficiently 



large n, 



Px~ (Gl) <e 



holds. 

Proof: Since (R,Aj) is achievable for X, there are an integer I and a code Ci such that 

(1//) log || <R + e (7) 

and 



4 j) 



(4 J \MX l ),Y 3 l ),Z^]<A 3 +e, Vj€j. (8) 
For each j G J", let fW> be a function on X 1 such that 

/W)(o») = 4 3} (a\d) - A, - e, a 1 G 
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Then, <[8J implies that 



f U) { X l ) 



< 0. 



Hence, the ergodic theorem guarantees the following fact: There exists n\ = ni(j,e,l,X) such that for any 
n > m there exists a set A n C X n satisfying (i) Pr{X n G .A^} > 1 — e and (ii) for any x n G A n , 

£ Pk(a l \x n )fU\a l )<e (9) 
where pi is the overlapping empirical distribution of i n defined as 



p;(a |.t ) = — — , a G A . 



n — fe + 1 
Note that pi and <7; ;s satisfy that 

l-l 

(n-l + l) Pl (a l \x n )=J2l(n-s)/l]q l;s (a l \x n ), a 1 G 

s=0 

and thus 



^(n ^ M"V)>yZ>;*(«V), a 1 eX l . (10) 



s=0 



By © and (0, for n > n x and G Ai, 



S=O a ! e ^l 1 ' 



Now, let S(j, n, x n ) be the set of all s such that 



Y <MaV)/°V) > 2J ( " (ID 

Then, by the Markov lemma, 

\S(j,n,x n )\<± 

for n > n\ and x" G A n . Further, let S(n, x n ) be the set of all s such that (fTTt holds for at least one j G J. 
Then, for n > 77,2 = maxj 7ii(j, e, £, X) and a;™ G An, we have 

|5(n >a! »)|<^ 

Thus, for n > 77,2 and a;™ G A n , there exists at least one s such that 

J2 m-AA* n )f U) {a l ) < 2J {n ~ l + 1) e, Vj G J. 

^— ' 77. — t) 

On the other hand, we can choose 77,3 such that for any 77 > 773, (n — I + l)/(n — I) < 3/2. Then, for any 

77 > 774 = max{ 77,2, 773} and x n G Ai> we have 

« ;J (a' 1^)^(0', CO < Aj + e+ 2J ( " "^A 
< Aj- + e + 3 Je 

<Aj-+4Je. (12) 

In other words, if n is so large that 77 > 77,4 and I < k n then for any x n G A we can choose Z, C/, and s 
satisfying (0 and ( fT2l . This means that _4„ C Hence, we have 

Px»(Q n ) > Px~{A n ) > 1 -e. 
This completes the proof of the lemma. □ 
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Fig. 1. Our coding system (J = 3). 
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Fig. 2. Wyner-Ziv coding when side information may be absent. 



Fig. 3. Complementary delivery. 
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Fig. 4. Coding system considered in |8 1. 
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